The present invention relates to a data network, and more particularly, relates to a completion queue management mechanism and an algorithm for checking on multiple completion queues and processing completion events in such a data network.
A data network is generally consisted of a network of nodes connected by point-to-point links. Each physical link may support a number of logical point-to-point channels. Each channel may be a bi-directional communication path for allowing commands and message data to flow between two connect nodes within the data network. Each channel may refer to a single point-to-point connection where message data may be transferred between two endpoints or systems. Data may be transmitted in packets including groups called cells from source to destination often through intermediate nodes.
In many data networks, hardware and software may often be used to support asynchronous data transfers between two memory regions, often on different systems. Each host system may serve as a source (initiator) system which initiates a message data transfer (message send operation) or a target system of a message passing operation (message receive operation). Examples of such a system may include host servers providing a variety of applications or services and I/O units providing storage oriented and network oriented IO services. Requests for work (data movement operations including message send/receive operations and remote direct memory access xe2x80x9cRDMAxe2x80x9d read/write operations) may be posted to work queues associated with a given hardware adapter, the requested operation may then be performed. It is the responsibility of the system which initiates such a request to check for its completion.
In order to optimize use of limited system resources, completion queues may be provided to coalesce completion status from multiple work queues belonging to a single hardware adapter. After a request for work has been performed by system hardware, notification of a completion event may be placed on the completion queue. Completion queues may provide a single location for system hardware to check for multiple work queue completions.
Completion queues may support two models of operation. In the first model, when an item is placed on the completion queue, an event may be triggered to notify the requester of the completion. This may often be referred to as an interrupt-driven model. In the second model, an item is simply placed on the completion queue, and no event may be signaled. It is then the responsibility of the request system to periodically check the completion queue for completed requests. This may be referred to as polling for completions.
However, completion queues are bound to a single hardware adapter of a host system in a data network, such completion queues cannot coalesce completions from work queues that reside on separate hardware adapters. Moreover, expensive locking or serialization routines are required to manage completion polling operations to multiple completion queues without starvation. Accordingly, there is a need for a single completion service to monitor multiple work queues in a host system with multiple hardware adapters while flexibly supporting both polling and event-driven completion operation models. Also needed is a performance-efficient completion queue management mechanism for checking multiple completion queues in the context of either a single thread or multiple threads of operation without the need for expensive locking or serialization techniques and processing completion events for optimal use of limited system resources.